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REMARKS 

Status of the Claims 

Claims 1, 3-12, 14, 16-26, and 28-32 and 34-42 are pending in the application, Claims 1, 3-4, 
6-8,. 10-11, 14, 16-26, 29, 31, 32, and 35-37 having been amended, new claims 35-37 having been 
added, and Claims 2, 13, 15, 27, and 33 having been previously canceled. These amendments further 
clarify the claims and also tend to place the claim set generally in a state similar to that presented 
when this application was first filed. Minor typographical errors have also been corrected. Please 
note that Claim 17 has NOT been canceled or "withdrawn from consideration," as was erroneously 
indicated under the "Disposition of Claims" section of the Office Action Summary page (form 
PTOL-326). Also note that new Claims 38-42 respectively generally correspond to the previously 
canceled Claims 2, 13, 15, 27, and 33. 
Amendment to the Title 

Please note that the title has been amended as set forth above, from "LOSSLESS 
MANIPULATION OF IMAGES," to "LOSSLESS MANIPULATION OF MEDIA OBJECTS." 
Claims Rejected under 35 U.S.C. § 103(a) 

The Examiner has rejected Claims 1, 3-12, 14, 16, and 18-30 as being unpatentable over Sull 
] et al. (U.S. Patent Application Publication No. 2002/0069218 hereinafter referred to as "Sull") and 
further in view of Wee et al. (U.S. Patent No. 6,553,150 hereinafter referred to as "Wee"). The 
Examiner asserts that it would have been obvious to one skilled in the art at the time of the invention 
to use the encoding process that indicates how each ICR (independent coding region) is to be "sliced" 
(see column 11, lines 10-32 of Wee) to modify editing of multimedia files (section 0002) of Sull 
because as detailed in column 11, this process results in optionally editing a portion of the bit stream 
without having to completely decode each entire area of interest as indicated by an optional process 
block 125. Applicants respectfully disagree with this reasoning and with the rejection of the claims 
for the reasons noted below. 

In the interest of reducing the complexity of the issues for the Examiner to consider in this 
response, the following discussion focuses on independent Claims 1, 14, 25, 31, 36, and 37. The 
patentability of each dependent claim is not necessarily separately addressed in detail. However, 
applicants' decision not to discuss the differences between the cited art and each dependent claim 
should not be considered as an admission that applicants concur with the Examiner's conclusion that 
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these dependent claims are not patentable over the cited references. Similarly, applicants' decision 
not to discuss differences between the prior art and every claim element, or every comment made by 
the Examiner, should not be considered as an admission that applicants concur with the Examiner's 
interpretation and assertions regarding those claims. Indeed, applicants believe that all of the 
dependent claims patentably distinguish over the references cited. However, a specific traverse of the 
rejection of each dependent claim is not required, since dependent claims are patentable for at least 
the same reasons as the independent claims from which the dependent claims ultimately depend. 
Discussion of Steps (a), (b), and(c) 

With regard to independent Claim 1, the Examiner asserts that Sull discloses applicants' step 
(a), which previously recited in an earlier amendment "accessing data defining the image to produce 
a representation of the image," and step (c), which previously recited in an earlier amendment 
"rendering a modified image in accord with the modification to the representation." The Examiner 
cites sections 0070, 0523, 0037, and 0081 of Sull in support of his assertion that the reference 
discloses step (a) and cites section 0165 of the reference in support of his assertion regarding step (c). 
These citations from Sull are reproduced below. : 

[0070] The present invention further provides a new approach to 'editing video 
materials, in which it only virtually edits the metadata of input videos to create a 
new video, instead of actually editing videos stored as computer files. In the present 
invention, the virtual editing is performed either by copying the metadata of a video 
segment of interest in an input metafile or copying only the URI of the segment into 
a newly constructed metafile. The present invention provides a way of playing the 
newly edited video only with its metadata. The present invention also provides a 
system for the virtual editing. The present invention can be applied not only to videos 
stored on CD-ROM, DVD, and hard disk, but also to streaming videos over a network. 
(Emphasis added, Sull, paragraph 0070.) 

[0523] The content analyzer 4508 analyzes the video, namely the scene of 
video frames, to find their type and purpose, the motion vector direction, and face/text, 
etc. Based on this information, the content selection module 4510 and the 
manipulation module 4514 transcode the video by selecting adaptively the attention 
area that is defined by a position and size for a rectangular window, for example, in a 
video that is intended to fit the size of the respective client display. The system 4500 
will select a dynamically transcoded (for example, scaled and/or cropped) area in the 
video without degrading the perceptibility of users. Also, this system has the manual 
editing routine that alters/adjusts manually the position and size of the transcoded area 
by the publisher and author. (Sull, paragraph 0523.) 
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[0037] When an image is transmitted to a variety of client devices with 
different display sizes, a scaling mechanism, such as format/resolution change, bit- 
wise data size reduction, and object dropping, is needed. More specifically, when an 
image is transmitted to a variety of client devices with different display sizes, a system 
should generate a transcoded (e.g., scaled and cropped) image to fit the size of the 
respective client display. The extent of transcoding depends on the type of objects 
embedded in the image, such as cards, bridges, face, and so forth. Consider, for 
example, an image containing an embedded text or a human face. If the display size of 
a client device is smaller than the size of the image, sub-sampling and/or cropping to 
fit the client display must reduce the spatial resolution of the image. Users very often 
in such a case have difficulty in recognizing the text or the human face due to the 
excessive resolution reduction. Although the importance value may be used to provide 
information on which part of the image can be cropped, it does not provide a 
quantified measure of perceptibility indicating the degree of allowable transcoding. 
For example, the prior art does not provide the quantitative information on the 
allowable compression factor with which the important regions can be compressed 
while preserving the minimum fidelity that an author or a publisher intended. The 
InfoPyramid does not provide either the quantitative information about how much the 
spatial resolution of the image can be reduced or ensure that the user will perceive the 
transcoded image as the author or publisher initially intended. (Sull, paragraph 0037.) 

[0081] The present invention also provides a novel scheme for generating 
transcoded (scaled and cropped) image to fit the size of the respective client display 
when an image is transmitted to a variety of client devices with different display sizes. 
The scheme has two key components: 1) perceptual hint for each image block, and 2) 
an image transcoding algorithm. For a given semantically important block in an 
image, the perceptual hint provides the information on the minimum allowable spatial 
resolution. Actually, it provides a quantitative information on how much the spatial 
resolution of the image can be reduced while ensuring that the user will perceive the 
transcoded image as the author or publisher want to represent it. The image 
transcoding algorithm that is basically a content adaptation process selects the best 
image representation to meet the client capabilities while delivering the largest content 
value. The content adaptation algorithm is modeled as a resource allocation problem to 
maximize the content value. (Sull, paragraph 0081.) 

[0165] Finally, the indexing methods of the present invention are enhanced by 
the unique modification of visual rhythm techniques that are part of other methods of 
the present invention. Modification of prior art visual rhythm techniques enable the 
system of the present invention to capture text information in the form of captions that 
are embedded into multimedia information, and even from video streams as they are 
broadcast, so that text information about the multimedia information can be included 
in the multimedia bookmarks of the present invention and utilized for storing, 
indexing, searching, retrieving, editing and rendering of the information. (Sull, 
paragraph 0165.) 
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However, note that the recitation of steps (a), (b), and (c) indicates at least that: (1) data that 
define the media object are accessed; (2) the data accessed are used to produce a representation of a 
media object, producing metadata that define the modification; and (3) a modified media object that 
is in accord with a modification to the representation is rendered. Thus, it can be appreciated from 
these steps that an important aspect of applicants' independent Claim 1 is that it recites the use of not 
only data but also metadata. Data is expressly recited in applicants' step (a), and metadata is 
specifically recited as being produced to define the modification. Step (d) recites that the modified 
image rendered in accord with the modification to the representation in step (c) is done with metadata 
that is stored in applicants' step (d). Also note that another important aspect, a temporal aspect of 
applicants' independent Claim 1, is that applicants' metadata does not come into existence until 
applicants' step (b) is performed, i.e. the metadata are produced when the modification is done in step 
(b). The modified media object is then rendered in accord with the modification to the 
representation, as recited in step (d), and the metadata define the modification that is applied when 
rendering the media object with the modification. 

In contrast, the citations to Sull provided by the Examiner fail to teach or suggest all of these 
aspects included in the recitation of step (a) - step (c). Note that Sull expressly discloses metadata in 
paragraph 0070, clearly indicating that the metadata in Sull are already in existence. Sull teaches in 
the bold font citation above that there is virtual editing of the metadata, and this editing is performed 
by copying the metadata of a video segment of interest that is in an input metafile. So, the metadata 
already exists in Sull and editing is performed on the existing metadata. This is not what applicants' 
are reciting in their claim steps (a) through (c). First, data are accessed that define the media object, 
and then, a modification is performed that produces metadata that are stored in step (d). Thus, Sull 
does not teach or suggest all of the recitation of steps (a) through (c). 

The Examiner has cited other portions of Sull in addition to paragraph 0070, in support of his 
assertion that Sull anticipates these claims. But, applicants note that Sull has expressly utilized the 
term "metadata" in explaining how a video file is edited. No mention of the term metadata appears in 
the other citations. Accordingly, it is apparent that only paragraph 0070 of Sull is relevant of all the 
citations, because that paragraph is the only one of the citations that even discusses metadata. For 
example, the discussion of paragraph 0523 of Sull simply discusses a generally conventional 
approach to automatically cropping a video frame to enable it to fit on a smaller display device, and 
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mentions in passing that a user can be provided the option to manually edit the automated cropping 

and position of the cropped portion of the video frame. There is no mention of metadata in 

paragraph 0523 of Sull, and thus, that citation is not particularly relevant to applicants' claims. 

Further Discussion of Steps (b) and (d) in Regard to Wee 

Significant differences exist between applicants' Claim 1 and the cited art because it does not 

appear the cited art teaches the claim recitation of selectively editing the representation of a media 

object and storing metadata without modifying the data that define the media object. 

With respect to independent Claim 1, the Examiner asserts that Wee discloses applicants' step 

(b) that previously recited in an earlier amendment "enabling a user to selectively edit the 

representation of the image by applying a modification to the representation, wherein the 

modification comprises the step of selectively cropping the representation," as well as step (d) that 

recited "storing metadata that define the modification applied to the representation in association with 

the image, without modifying the data that define the image, said metadata defining a selected size 

and a selected position of a crop outline on the representation of the image that is provided to indicate 

limits of a cropped image." The Examiner cited column 30, lines 30-53 in support of his assertion 

regarding step (b) and cited column 11, lines 10-32 in support of his assertion regarding step (d). 

These citations are reproduced below: 

The second image or sequence 507 is modified as desired by the user; 
preferably, the software affords the user the opportunity to view the second image or 
sequence overlaid upon the first image, and to crop, scale and change the perspective 
of the second image. Accordingly, the second image or sequence is adjusted by the 
user to be roughly compatible with its destination within the video input, as indicated 
by reference numerals 513 and 515. The software takes the second image or sequence 
and maps that information as modified by the user to the dimensions of the decoded 
spatial regions, and fits substitute image data into the original regions of the video 
input (or at least within a selected object of the original regions). Preferably, the user 
is afforded the opportunity (via visual display, zoom and a user interface such as a 
mouse) to further crop the substitute image data, or to make that data translucent over 
the original image data. The software also preferably affords the user menu functions 
to fit accelerated or decelerated display of the second image (sequence) with respect to 
the original image data (e.g., via frame dropping), or to reverse play of the second 
image (sequence) with respect to the video input, or to splice several image sequences 
together to form replacement image data for a region. These functions are indicated by 
reference numerals 517, 519, 521 and 525. (Wee, column 30, lines 30-53.) 
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Once regions have been selected by the user and defined over the desired 
frame range of interest, the image processing system then encodes (or transcodes) the 
selected range of video, as indicated by master function block 115. During this 
process, if prior compression data is available for data which has been unaltered, that 
data is preferably re-used to save time that might otherwise be required in computing 
new compression data for unaltered regions, as indicated by block 117. Details are 
described below for one system that permits image processing software to track 
exactly which image frames have altered and in which manner alteration has been 
occurred. As part of the preferred encoding process and format, a region map can be 
developed for each frame of a group of pictures ("GOP") which indicates how each 
ICR is to be "sliced." The GOP is then compressed according to these region maps, 
and the result of this process ideally is a bit stream with independently coded regions. 
These operations are indicated, respectively, by blocks 119, 121 and 123. The bit 
stream is then suitable for storage or transmission, and is inherently suited for optional 
editing a portion of the bit stream without having to completely decode each entire 
frame of interest, as indicated by an optional process block 125. (Wee, column 11, 
lines 10-32.) 

However, note that the recitation of applicants' step (b) in Claim 1 indicates that: (1) a user is 
enabled to selectively edit the representation of the media object; and (2) this editing is carried out by 
making a modification to the representation, producing metadata that define the modification. Wee 
does not teach or suggest the step of selectively editing the representation of a media object. The 
Examiner has not indicated which elements in the above citation to Wee that he believes are 
equivalent to applicants' claim recitation. However, applicants recite editing the representation (e.g., 
a display) of a media object (e.g., a first image) and then modifying the representation (e.g., a 
displayed image) by selectively editing it, producing metadata that define the modification. Wee 
recites in column 30, lines 4-20 that it may be desired to replace, for example, a graphic showing a 
billboard with a second image. Specifically, Wee recites the example of replacing an English 
language sign (in a first image of the billboard) with a Chinese language equivalent in a second 
image. In contrast, applicants recite in step (b) that it is the representation (e.g., display) of the media 
object (e.g., an image), not a second image that a user is enabled to selectively edit. This 
modification to the representation (e.g., an image), might include the step of cropping the 
representation if the media object is an image. In contrast, Wee teaches that the user is able "to crop, 
scale and change the perspective of the second image. " Thus, Wee is teaching cropping a second 
image; Wee is not teaching selectively editing the representation of a display of a single media object, 
which would correspond to the first image in Wee. 
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Applicants' step (d) recites at least three aspects of the claim pertaining to metadata, 
including: (1) metadata are stored in association with the media object; (2) the metadata define the 
modification applied to the representation of the media object; and (3) the metadata do not modify the 
data defining the media object. The Examiner has not indicated which elements he believes are 
equivalent to applicants' claim recitation in regard to these three aspects of the claim. If the 
Examiner believes that an independently coded region of the selected range of video is equivalent to 
metadata, then applicants note that it does not appear that Wee teaches that the ICR (which is 
presumably what the Examiner believes corresponds to applicants' recited metadata) do not modify 
the data defining the media object. Wee appears to teach that a permanent alteration to the data 
defining the media object has taken place. Specifically, Wee gives the example of a desire to edit a 
region, for example, to change the color of a particular actor's shirt. Wee teaches that this color 
correction may be accomplished by inserting a color corrected signal that is re-encoded into bit 
stream format and inserted in the place of the original bit stream data for the set of actors (Wee, 
column 9, lines 15-21). 

Accordingly, the rejection of independent Claim 1 under 35 U.S.C. § 103(a) over Sull and 
further in view of Wee should be withdrawn, for the reasons given above, since the combined cited 
references do not teach or suggest all of the recitation of independent Claim 1. 

Claims 3-12 ultimately depend from independent Claim 1. Because dependent claims 
inherently include all of the steps or elements of the independent claim from which the dependent 
claims ultimately depend, dependent Claims 3-12 are patentable for at least the same reasons 
discussed above with regard to independent Claim 1. Accordingly, the rejection of dependent 
Claims 3-12 under 35 U.S.C. § 103(a) over Sull and further in view of Wee should be withdrawn. 
Discussion of the Rejection of Independent Claim 14 

Independent Claim 14 is directed towards a system for lossless editing of a media object and 
subparagraphs (d)(i) through (iv) are similar to steps (a) through (d) of independent Claim 1. 
However, this time the Examiner has cited Sull only in regard to his rejection of subparagraph (d)(i) 
and has cited Wee in regard to his rejection of subparagraphs (d)(ii), (d)(iii), and (d)(iv). Therefore, 
applicants' comments traversing the rejection of independent Claim 1 are equally applicable to show 
the patentability of applicants' independent Claim 14. 
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Accordingly, the rejection of independent Claim 14 under 35 U.S.C. § 103(a) over Sull and 
further in view of Wee should be withdrawn, for the reasons given above, since the combined cited 
art does not teach or suggest all of the recitation of independent Claim 14. 

Claims 16 and 18-24 ultimately depend from independent Claim 14. Because dependent 
claims inherently include all of the steps or elements of the independent claim from which the 
dependent claims ultimately depend, dependent Claims 16 and 18-24 are patentable for at least the 
same reasons discussed above with regard to independent Claim 14. Accordingly, the rejection of 
dependent Claims 16 and 18-24 under 35 U.S.C. § 103(a) over Sull and further in view of Wee 
should be withdrawn. 

Discussion of the Rejection of Independent Claim 25 

Independent Claim 25 is directed towards a method for lossless modification of a media 
object. Applicants respectfully disagree with this rejection, at least because Sull and Wee neither 
teach nor suggest applicants' step (h), contrary to the Examiner's assertion. The differences between 
the recitation of the claim and the cited references that are discussed above in regard to applicants' 
Claim 1 also apply in traversing the rejection of Claim 25: In addition, there are further differences 
between this claim and the cited art, as discussed below. 

The Examiner asserts that Wee teaches applicants' step (h), which recites "storing metadata 
that now define the second modification of the image, without modifying the data that define the 
image," and the Examiner cites column 11, lines 10-32 of Wee in support of this assertion. But 
again, as noted above in connection with independent Claim 1, the original bit stream in Wee appears 
to be permanently edited since a signal is re-encoded into bit stream format and inserted in the place 
of the original bit stream data (Wee, column 9, lines 17-20). Also, Wee does not appear to teach or 
suggest a second modification, producing metadata that define the second modification, wherein the 
metadata that define the second modification are stored. 

Accordingly, the rejection of independent Claim 25 under 35 U.S.C. § 103(a) over Sull and 
further in view of Wee should be withdrawn, for the reasons given above, since the combined cited 
art do not teach or suggest all of the recitation of independent Claim 25. 

Claims 26, and 28-30 ultimately depend from independent Claim 25. Because dependent 
claims inherently include all of the steps or elements of the independent claim from which the 
dependent claims ultimately depend, dependent Claims 26 and 28-30 are patentable for at least the 
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same reasons discussed above with regard to independent Claim 25. Accordingly, the rejection of 
dependent Claims 26 and 28-30 under 35 U.S.C. § 103(a) over Sull and further in view of Wee 
should be withdrawn. 
Claims Rejected under 35 U.S.C. § 102(e) 

The Examiner has rejected Claims 31-32 and 34-37 as being anticipated by Sull. The 
Examiner asserts that Sull describes each element of applicants' claimed invention. Applicants 
respectfully disagree for the reasons noted below. 
Discussion of the Rejection of Independent Claim 31 

Independent Claim 31 is directed towards a system for lossless modification of a media object 
and generally corresponds to independent method Claim 25. Thus, as explained above in connection 
with independent Claim 1, which recites steps similar to step (c)(i) and step (c)(iii), Sull does not 
teach or disclose metadata in a manner equivalent to that of applicants' recited claim steps. In 
addition, it also follows that the cited reference does not teach step (c)(viii) of Claim 31, because 
metadata asvdefined by Sull exists before any modifications are performed and are not produced as 
result of a modification. Further, the metadata that define the modification are not stored by Sull, as 
recited in this claim. 

Accordingly, the rejection of independent Claim 31 under 35 U.S.C. § 102(e) over Sull should 
be withdrawn, for the reasons given above, since Sull does not teach all of the recitation of 
independent Claim 31. 

Claims 32 and 34-35 ultimately depend from independent Claim 31. Because dependent 
claims inherently include all of the steps or elements of the independent claim from which the 
dependent claims ultimately depend, dependent Claims 32 and 34-35 are patentable for at least the 
same reasons discussed above with regard to independent Claim 31. Accordingly, the rejection of 
dependent Claims 32 and 34-35 under 35 U.S.C. § 102(e) over Sull should be withdrawn. 
Discussion of the Rejection of Independent Claim 36 

Independent Claim 36 is directed towards a machine readable medium having a data structure 
for lossless modification of a media object. The Examiner asserts that applicants' step (a) is 
disclosed by paragraph 0165 with the addition of paragraph 0523 disclosing the cropping element of 
applicants' claim limitation. In addition, the Examiner asserts that the applicants' step (b) is 
disclosed by sections 0070, 0165, and 0523. However, there is no teaching in Sull of a data structure 
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that stores both metadata and the data defining the image. Note that data defining a media object 
that is used to produce a representation are not the same as the metadata produced to define the 
modification of the media object. 

Accordingly, the rejection of independent Claim 36 under 35 U.S.C. § 102(e) over Sull should 
be withdrawn, for the reasons given above, since Sull does not teach all of the recitation of 
independent Claim 36. 

Discussion of the Rejection of Independent Claim 37 

Independent Claim 37 is directed towards a machine-readable medium having a data structure 
for a collection of media objects comprising a substorage. For the reasons presented above in 
connection with independent Claim 36, it does not appear that Sull discloses a data structure that 
stores both metadata that define a modification and the data defining a media object. 

Accordingly, the rejection of independent Claim 37 under 35 U.S.C. § 102(e) over Sull should 
be withdrawn, for the reasons given above, since Sull does not teach all of the recitation of 
independent Claim 37. 

In view of the Remarks set forth above, it will be apparent that the claims remaining in this 
application define a novel and non-obvious invention, and that the application is in condition for 
allowance and should be passed to issue without further delay. Should any further questions remain, 
the Examiner is invited to telephone applicants' attorney at the number listed below. 



I hereby certify that this paper or fee is being deposited with the United States Postal Service "Express Mail Post Office to Addressee" 
service under 37 C.F.R. § 1.10 on the date indicated above and is addressed to the Commissioner for Patents, Alexandria, Virginia 



Respectfully submitted, 




Sabrina K. Maclntyre 
Registration No. 56,912 



SKM/RMA:elm 



Express Mail Label No. EV538087416US 



Date of Deposit October 17, 2005 



22313-1450. 



(Name of person mailing paper/fee) 



Elizabeth L. Miller 




MS # 192036.01 

MICR0259-1-1N0259 AMPrelim 10-17-2005.doc 



-19- 



LAW OFFICES OF RONALD M. ANDERSON 
600 - 108th Avenue N.E., Suite 507 
Bellevue, Washington 98004 
Telephone: (425)688-8816 Fax: (425)646-6314 



